A Unified Lyapunov Framework for Finite-Sample Analysis of Reinforcement Learning Algorithms

Reinforcement learning (RL) is a paradigm where an agent learns to accomplish tasks by interacting with the environment, similar how humans learn. RL therefore viewed as promising approach achieve artificial intelligence, evidenced remarkable empirical successes. However, many algorithms are theoretically not well-understood, especially in setting function approximation and off-policy sampling employed. My thesis [1] aims at developing thorough theoretical understanding performance of various through finite-sample analysis. Since most essentially stochastic (SA) for solving variants Bellman equation, first part dedicated analysis general SA involving contraction operator, under Markovian noise. We develop Lyapunov we construct novel called generaled Moreau envelope. The results on enable us establish bounds tabular (cf. Part II thesis) when using III thesis), which turn provide insights several important problems community, such efficiency bootstrapping, bias-variance trade-off learning, stability control. main body this document provides overview contributions my thesis.